Seems that the lint fails even for the main https://app.circleci.com/pipelines/github/facebookresearch/hydra/14763/workflows/e495a59a-f45c-4531-a51a-898dad566d50/jobs/146566

@Jasha10

Thank you for your comment!

Do you expect that this change to the optuna sweeper plugin will be backwards compatible for Hydra users?

If the meaning of the backwards compatibility here is that we can run hydra with both Optuna v2.x and v3.x, we should not merge this PR. This code change can run only with Optuna v3.x.

As Optuna will keep the backwards compatibility for a while, e.g. until v4, we do not need to merge this PR at the moment. But, in that case, one note is that someone needs to merge this PR when Optuna will totally break the backwards compatibility.

Do you expect that this change to the optuna sweeper plugin will be backwards compatible for Hydra users?

If the meaning of the backwards compatibility here is that we can run hydra with both Optuna v2.x and v3.x ...

My question is about the Optuna Sweeper's user API: will Hydra users need to update their yaml configs?
I think the answer is no; users will not need to update anything.

This code change can run only with Optuna v3.x.

This is fine :)
If users need Optuna v2.x, they can use an old version of the optuna sweeper plugin.
Edit: We are looking into usage patterns to see whether Optuna v3.x is as popular as v2.x. If not, we might delay merging this PR for a while...

I will push some commits to this PR branch to try and make the CI pass.

Do you expect that this change to the optuna sweeper plugin will be backwards compatible for Hydra users?

If the meaning of the backwards compatibility here is that we can run hydra with both Optuna v2.x and v3.x ...

My question is about the Optuna Sweeper's user API: will Hydra users need to update their yaml configs?
I think the answer is no; users will not need to update anything.

Your understanding is correct. They do not need to change it.

This code change can run only with Optuna v3.x.

This is fine :)
If users need Optuna v2.x, they can use an old version of the optuna sweeper plugin.
Edit: We are looking into usage patterns to see whether Optuna v3.x is as popular as v2.x. If not, we might delay merging this PR for a while...

Thank you! You can merge it when it is appropriate.

It is possibile to install using pip a dev version of hydra sweeper with support for optuna 3 ?

Could this please be merged? The 2.1 version of Optuna is very much out-of-date.

I'd also like this to be merged!

@Jasha10
Long time no see and thank you for working on it!
Should I do anything to merge this PR to fix CI?

Hi @keisuke-umezawa,

Since it's been a while, it would be good to rebase this diff on top of the main branch.

Currently I'm no longer an official maintainer of Hydra. That being said, I do hope this diff can be merged at some point.

@AlexLup

It is possibile to install using pip a dev version of hydra sweeper with support for optuna 3 ?

You can pip-install the version of hydra-optuna-sweeper from this PR

pip install 'git+https://github.com/keisuke-umezawa/hydra/@feature/fix-optuna-v3#egg=hydra-optuna-sweeper&subdirectory=plugins/hydra_optuna_sweeper'

Or you can git-clone the repo, check out @keisuke-umezawa's feature/fix-optuna-v3 branch, and run pip install --editable . in the plugins/hydra_optuna_sweeper subdirectory of the repo.

@Jasha10
Sorry for being late. I rebased it against the current main branch.
Who should I talk with to merge this PR?

No problem @keisuke-umezawa. I believe that @omry and @shagunsodhani are current maintainers of Hydra.

I believe the CI failures here are unrelated to this diff. Once #2692 and #2693 are fixed, it should be possible to get the CI green.

Are there any updates on this? It seems the issues mentioned by @Jasha10 are all merged. I would love to use optuna >= 3 .

Are there any updates on this? It seems the issues mentioned by @Jasha10 are all merged. I would love to use optuna >= 3 .

In the meantime you should be able to install a version with the fix with the command in #2360 (comment)

I would like to bump this PR :)
Lack of optuna>=3 starts to become problematic since introduction of new pandas version, I need to have 2 separate envs to extract my data (with pandas) and one for actuall sweeps with older version of SQLAlchemy (optuna<3 relies on older SQLAlchemy incompatible with new pandas)

Can I help with any requested changes?